Report of the 
Board of Regents 



National Library of Medicine 
Long Range Plan 



Electronic Imaging 



Voioifioiiioioioioiiifxoioioiiii] 
fSifflM. >loniMoioitioioi*mwxuni( 

A!?itff^dVD ^Pioioioioioieiiii 

^?!!?}!Hi9K^^B^^V viomoiiiiuioo 

fSif}i"W J ^^^F . 101010X11111100 
t A Vioioioioioiiii 

fXJ?i4J}i?f« T^^K # uoioiiiiiiioo 

V^^V # foioioioiouuif 
;S2?}HH?}fm / ^loiouuiiiooo 

JS?l}??}fiJl!^ # |oioioioioiiiii 

^fSifJiJ}J?J?!^>^ / fioioiiiiiuooo 
JXXHXXJJ}?}}^^ poioioioioiiiii 
lS?JJ??}?ii^**^ Ao ioiQioioiiiii 
^^^^■■■■Bio^oiiiiiiiooo 
lOOllMT^ ^^^^^^^^^^^Hioioioioiiui 

0101111111000 
llOlOlOlOlllll 

rioioiiuiiiooo 
iioieioiiiii 



1 

001 
1001 
001011 
10011001 
1001100101 
001011111010 
100110010111 
0010111110101 
1001100101111 
00101111101 

looiiooioir 

10011001011 
QOiftnnini 



NATIONAL INSTITUTES OF HEALTH 
National Lbrary of Medicine 



National Library of Medidne Cataloging in Publication 

Z National Library of Medicine (U.S.)- Board of Regents. 

675.M4 Electronic imaging : report of the Board of Regents. — 

N2782d Bethesda, Md. : U.S. Dept. of Health and Human Services, 

Public Health Service, National Institutes of Health, 1990. 

"National Library of Medicine long range plan." 
"April 1990." 



1 . National Library of Medidne (U.S.) 

2. Image Processing, Computer- Assisted. 

I. Title. II. Title: National Library of Medicine long range plan. 



Report of the 
Board of Regents 



National Library of Medicine 
Long Range Plan 



Electronic Imaging 




NIH PublicaHon No. 90-2197 



U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES 
Public Health Service 
National Institutes of Health 
April 1990 



Foreword 



The 1987 Long Range Plan of the Board of Regents* recommended that the 
National Library of Medicine should "...thoroughly and systematically investigate 
the technical requirements for and feasibility of instituting a biomedical images 
library." It encouraged NLM to consider building and disseminating medical 
image libraries much the same way it acquires, indexes, and provides access to the 
biomedical literature. Early last year under the direction of the Board of Regents, 
an ad hoc planning panel was convened to explore in-depth the proper role for 
NLM in this rapidly changing field. 

The Panel was chaired by Dr. Donald West King of the University of Chicago, 
and represented both the producers and potential users of biomedical images in 
medical education, research and clinical practice. In charging the Panel, the Board 
wished to know whether such image libraries would be used by students, faculty 
and clinicians; what standards and data exchange arrangements would be needed; 
and whether the proposed 3-D Visible Human Project, a very compelling idea that 
emerged from an earlier NLM workshop, would be a good place to begin. 

On January 18, 1990, the Board of Regents reviewed the draft report prepared by 
the Panel and unanimously approved the incorporation of its recommendations as 
part of the NLM Long Range Plan. The Board applauded the leadership of Dr. 
King and commended Dr. Daniel R. Masys, Director of the NLM's Lister Hill 
National Center for Biomedical Communications, for his outstanding assistance 
to the work of the Panel. This report and the recommendations contained therein, 
presents exciting new challenges to the National Library of Medicine in advancing 
our fundamental knowledge and use of computerized representations of biomedical 
structural data, and its transmission across national networks in support of the 
needs of the biomedical community. 

Don E. Detmer, M.D. 

Chairman 

Board of Regents 

National Library of Medicine 

♦National Library of Medicine, Long Range Plan; report of the NLM Board of Regents. Bethesda, 
Md.: National Library of Medicine, January 1987. 



Preface 



The concept of building electronic image libraries, and providing for their 
transmission across high speed digital networks, presents a special opportunity for 
biomedical science in general and for the National Library of Medicine in 
particular. Much of our understanding of complicated processes of health and 
disease lies in images, pictures of body systems, organs and molecules which 
cannot effectively be described in words. The NLM Planning Panel on Electronic 
Imaging has defined a role for the Library which capitalizes on its unique 
technical capabilities, its very considerable stature in the biomedical community, 
and its position as a government agency with a Congressionally-mandated mission 
to acquire and disseminate biomedical knowledge ~ in all its forms. 

I am indebted to the serious discussion and diligent work on the part of the entire 
Panel membership during its two meetings in Bethesda, and its between-meeting 
deliberations. I thank them on behalf of the Library for their excellent 
contributions. I am also grateful for the support of Dr. Donald A.B. Lindberg, 
NLM Director; the staff of the Office of Planning and Evaluation, its Director, Dr. 
Elliot R. Siegel, and Ms. Susan B. Slater who served as Executive Secretary to the 
Panel; and to the other capable NLM staff who assisted us. 

Donald West King, M.D. 
Chairman 

Planning Panel on Electronic Imaging 
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Executive Summary 



Background Images are an important part of biomedical knowledge. Pictures 

facilitate the understanding of biological structure and function, 
and are an essential component of education, research, and health 
care delivery. New computer-based technologies are providing 
an unprecedented opportunity to supplement the traditional two 
dimensional images of medicine, such as pictures in textbooks 
and plain radiographs, with dynamic three dimensional images. 
These images can be viewed, rotated, and reversibly dissected in 
a manner analogous to the physical objects they represent, 
providing valuable instruction to the student, insight to the 
researcher, and critical treatment planning information to the 
practitioner. 

The National Library of Medicine (NLM) has long been a world 
leader in the archiving and distribution of the print-based images 
of biology and medicine. NLM has also been a pioneer in the 
use of computer systems to encode and distribute the textual 
knowledge of the life sciences. The NLM's Long Range 
Planning effort of 1985-86 foresaw a coming era where NLM's 
bibliographic and factual database services would be 
complemented by libraries of digital images, distributed over 
high speed computer networks and by high capacity physical 
media. This planning panel was convened to recommend when 
and how the NLM might proceed in the development of such 
digital image libraries. 

Findings Acquiring and providing access to digital image libraries is 

entirely consistent with NLM's institutional mandate to acquire, 
organize, and make available the knowledge of biology and 
medicine. 

The technologies underlying the computer-based representation 
and display of complex three dimensional biological structure are 
sufficiendy mature that the NLM can proceed with the building 
of prototype digital image libraries. 

The digital image requirements of education, research and 
clinical practice differ from one another. Education may 
profitably employ established teaching collections of 
representative normals and abnormals. Research requires 
laboratory tools which empower an investigator to make and test 
hypotheses based on numerical, conceptual and image data, to 
share that data easily with collaborators, and contribute it to 
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shared national resources when appropriate. Clinical practice 
imaging is often focussed on the diagnostic and treatment 
planning considerations for a single patient, and uses images 
representing that patient's unique condition. 

There remain fundamental research problems in the domain of 
computerized representation of biomedical structural data, and its 
linkage to related text and numeric data. Additional research 
support is needed to facilitate progress in this emerging 
subdiscipline of medical informatics. 

NLM connection to and use of high speed computer networks is 
an essential prerequisite for the efficient distribution of 
computer-based digital images. 

Recommendations The panel makes five principal recommendations: 

1. NLM should undertake a first project, building a digital 
image library of volumetric data representing a complete 
normal adult human male and female. This "Visible 
Human" project would include digital images derived 
from computerized tomography, magnetic resonance 
imaging, and photographic images from cryosectioning of 
cadavers. A working group should be assembled from 
experts in anatomy, clinical imaging, and computer 
science to estabhsh standards for acquisition, and 
computer representation of the data. Technical guidelines 
for this phased project are offered. The panel views this 
project as a cornerstone for a future set of related image 
libraries, and a test platform for developing methods and 
standards. 

2. The NLM should support a follow-on research effort to 
develop methods, tools and standards for classification of 
anatomic image data from the Visible Human Project, so 
that applications may be developed which can extract, 
manipulate and display image subsets on the basis of 
organs, tissues, body systems and biologic function. 

3. NLM should expand upon initial image libraries 
composed of normal structure to encompass specialized 
image collections which represent related structural 
information, such as embryological development, normal 
and abnormal variations, and disease -related images. In 
this regard, NLM should collaborate with appropriate 
professional societies and other organizations to identify 
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and pursue worthy examples of specialized image 
collections which have been developed by subject 
experts. 

4. NLM should encourage and support investigator-initiated 
research into methods for representing and linking spatial 
and textual information (and other relevant datatypes), 
and support efforts to introduce computer reconstructed 
anatomical imaging technologies into health professions 
curricula nationwide. 

5. The NLM should develop and enhance its wide area 
computer network connections to provide an efficient 
electronic distribution mechanism for large digital files 
such as those encoding biomedical images. In the 
development phase of imaging projects, NLM should 
continue and enhance its connectivity to NSFnet and the 
research Internet. NLM should consider developing 
image-based applications which make use of the proposed 
gigabit speed National Research Network. 

Resources In the short term, the costs of undertaking the data 

acquisition phase of the Visible Human project can be 
derived from existing technologies and similar projects 
undertaken on a smaller scale. The panel estimates that 
Phase I of this project would require approximately $1 
million and yield a uniquely valuable image data set. 

Phase n of the Visible Human Project-the classification 
of the image data-is considerably more labor and time 
intensive if undertaken using current manual or semi- 
automated methods, such as contour tracing by skilled 
anatomists. Between 40 and 50 man-years of effort 
would be required, at a cost of approximately $5 million, 
to fully classify the anatomic data. 

Research to develop new methods for representing and 
linking structural and symbolic data in the life sciences 
would benefit greatly from support for 5-10 high quality 
investigator-initiated grants annually at an estimated 
annual cost of $3 million. 



NLM connection to high speed computer networks is an 
extension of its current connections to NSFnet and the 
research Internet through the Lister Hill Center. The 
panel believes that over the next several years 
incremental improvement in bandwidth should be 
available to the NLM at approximately the same costs as 
it now bears (approximately $100,000 per year). 
Experimental projects involving commercial 
telecommunications networks should be pursued under 
collaborative cost-sharing agreements. 
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Background 

The visual nature Human vision provides an extraordinarily powerful and effective 

of biomedical knowledge means for acquiring information. Much of what we know about 

ourselves and our environment has been derived from images 
processed by various instruments, ranging from microscopes to 
telescopes which extend the range of human vision into realms 
beyond that which is naturally accessible. 

In the life sciences, the relationships between biological form 
and function have been central to the understanding of health 
and disease throughout recorded history. Indeed, illustrations of 
anatomy are the most enduring and informative relics of a 
medieval medical science whose theory has long since been 
discarded. An intimate understanding of biological structure and 
its implications for therapy remains a sine qua non for medical 
specialties such as surgery, neurology, and radiology, as well as 
a learning challenge for all health professionals. Until recendy, 
a fundamental limitation in the teaching and understanding of 
three dimensional structure was that media for reproducing 
images was two dimensional, as exemplified by the printed page 
and the plain film radiograph. The attempt to convey three 
dimensional content by presentation of two dimensional views 
forces upon the viewer an exercise in mental model construction 
which can be difficult and possibly fraught with inaccuracy; it 
endures as one of the major hurdles in the curricula of all health 
professions. For the trained viewer, who has long experience in 
the "intuitive" correlation and interpretation of 2-D sections with 
3-D images, it is hard to determine how and under what 
circumstances such expertise was developed, so that it can be 
analyzed and transmitted in a more systematic manner to the 
practitioner and scientist in training. 

Early three dimensional photographic technologies were brought 
to bear on the problem of human anatomy. The Bassett 
collection of anatomy images used the popular Viewmaster 
technology of matched stereo pairs of color transparencies to 
convey the essential qualities of depth and spatial orientation of 
gross anatomic structure. Still, the user of such photographic 
collections is limited to the static content of the images acquired 
at the time of image capture. No facility exists to move the 
object of study, to change the perspective of one's viewpoint at 
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will in a manner analogous to having the physical object at hand, 
or to dynamically "enter" an object to disclose its internal 
representation. 

New technologies The emergence of the digital computer has provided a new 

technology for the acquisition, storage, manipulation, and display 
of complex images. Central to the capabilities of digital image 
display has been the development of large memory computers, 
specialized graphics processors, and high speed digital networks, 
along with robust and economical solid state scanning, high 
resolution display devices, and digital optical disks. 

Pictures can be produced by computer in two fundamentally 
different ways\ The first is the representation of pictorial 
elements as geometrical objects, such as a collection of polygons 
whose shape and surface quaUties can be calculated to produce a 
likeness of a real world object; the underlying representation is 
based on mathematical formulas for the geometrical primitives 
which combined make up the complete object. 

The second method of picture representation by computer is 
sample based (also called pixel-based or image-based). In this 
method, the pictorial data may be thought of as individual points 
(pixels) in a large two dimensional field, which taken together 
portray an object recognizable by the viewer. In this scheme, 
the computer generally does not have an independent 
representation of objects within the image. 

The distinction between the two picture representation methods 
is crucial to computer-based biomedical imaging. Most clinical 
images are acquired using sample-based methods. Thus, 
although the varying x-ray patterns produced by the 
computerized tomographic (CT) scanner can be rendered into a 
display which a physician interprets as various anatomical 
structures, to the computer the pixel components of the image 
are not manipulable as objects (e.g. internal organs) except to the 
extent that some tissues share a unique and homogenous 
radiodensity (e.g., bone). Yet for purposes of understanding the 
content of complex medical images, the identification of objects 
and object boundaries within sample-based images is essential. 

Classification, i.e., the assignment of a sample or pixel to an 
organ, is a fundamental and difficult problem. The problem is 
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typified by considering a CT sample that happens to fall at the 
boundary of bone and muscle tissue. The sample at that point 
represents some combination of both tissue types. Many 
classification techniques used today class such a pixel as one or 
the other tissue, thus introducing inaccuracies into the objects 
extracted from the sample set. Problems such as these may be 
overcome by better application of sampling methods, but the 
classification of objects within pixel sets remains a daunting 
problem. 

NLM Long Range Plan In 1985, the NLM commissioned a long-range planning effort 

involving over 100 experts in the fields of computer and library 
science, health professions education, and medical informatics 
research. Charged with looking five, ten and twenty years into 
the future, the planning panels identified major trends, 
impediments to progress, and "windows of opportunity" for the 
NLM to accommodate an information-rich future, or make 
elements of that future happen if appropriate to NLM's 
institutional mission. 

Not surprisingly, the planning panels saw an increasing role for 
electronically-represented images in clinical medicine and 
biomedical research. As the world's foremost archive and 
distribution center for biomedical knowledge, the NLM was 
encouraged by the planning panel on Medical Education to 
explore the feasibility of building and making available 
electronic image libraries, much the same way that it maintains, 
indexes, and provides access to the biomedical literature^. 

However, the technologies to support such image libraries, 
associated high bandwidth communications, access and 
increasingly high resolution representation are new, and evolving 
rapidly in a number of disparate directions with heterogenous 
hardware and software requirements, and few if any standards 
for sharing of image data. To a public institution such as the 
Library, this flux translates into risk that prematurely launched 
programs will incur significant expense, yield image library data 
sets that are useable by only a small fraction of its constituency 
(or none at all), and be superseded by later developments which 
render its work obsolete and perhaps untransformable to new 
standards. 
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Current NLM digital Research and development involving digital images has 

imaging projects been ongoing within the NLM's Lister Hill National Center for 

Biomedical Communications for several years. Appropriate to 
the Library's mission to serve as a national archive for 
biomedical information, investigations in the use of digital bit- 
mapped graphics to preserve page images of published material 
have been pursued in an Electronic Document Storage and 
Retrieval (EDSR) Program. The basic model for this research is 
the acquisition of high resolution (2,000 by 2,000 pixel) 
electronic images from biomedical texts, image enhancement and 
compression by computer algorithms, storage of page images on 
optical disk, and retrieval/display of page images linked to 
searches of NLM's bibUographic databases. The EDSR 
investigations have included extensive system software 
development and cost/throughput models to compare electronic 
archiving with more traditional modes of literature archiving 
such as microfilm. 

An ambitious follow-on to the EDSR project is just beginning. 
Called the Machine Readable Archives in Biomedicine (MRAB), 
this project explores elements of document image 
"understanding", using automated algorithms to segregate text 
from graphics, and omnifont character recognition to convert 
printed text to machine readable character streams, while 
preserving nontextual regions in bitmapped form. 

Color and monochrome digital representations of complex 
medical images such as microscopic tissue sections, 35mm 
dermatology slides, and clinical radiographs provide the focus 
for other Lister Hill Center projects. A key element of this 
research is the development of methods for automated assembly 
of overlapping digital image "tiles" into a composite high 
resolution image. 

Innovative research on computer-based representation of 
three dimensional anatomic data is underway in a number of 
centers nationwide. In June of 1988 the National Library of 
Medicine convened representatives from eight academic medical 
centers involved with production and use of computer-based 
three dimensional anatomic imaging. The intent of this meeting 
was to review the current state of the art in 3-D imaging 



1988 3-D Anatomical 
Reconstruction 
Workshop 
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techniques, and to identify possible contributions that the Library 
might make to this field as a public agency. 

Each of the university groups presented an overview of the types 
of activities at their own centers. In the aggregate, it became 
clear that the power, graphical display capabilities, and 
affordability of current computers are sufficient for many 
educational and research applications in 3-D anatomical imaging, 
and that there are unique merits to images rendered from 
digitized anatomic databases. The most dramatic of these is the 
ability to isolate, highlight, "reversibly dissect", rotate, and view 
from multiple angles single and grouped tissues, organs, body 
regions, and physiologic systems. Anecdotal evidence of the 
enthusiastic acceptance of 3-D imaging, especially for complex 
anatomic systems such as the central and peripheral nervous 
system, and the use of 3-D images in surgical planning, was 
presented. 

The meeting participants pointed to the time and labor-intensive 
qualities of anatomic data acquisition as significant impediments 
to the wider use of 3-D anatomic reconstruction data in health 
professions education, treatment, research, and patient education. 
No spatial data set of anatomic coordinates for the complete 
human body exists in the public domain, and the project to 
develop such a data set would require more resources than any 
single academic group could reasonably devote to it. 

Accordingly, it was the consensus recommendation of the group 
that the Library could contribute substantially to the 
advancement of the field by supporting the development of an 
image data set of an entire human male and female. This 
"Visible Human Project" would be carried out in two phases. 
The first would be acquisition of enhanced computed 
tomography images of representative, carefully selected and 
prepared male and female cadavers. The original cadaveric 
material would be preserved. The second component of the 
initial image acquisition would involve selective cryosectioning 
of the same cadavers used for development of the CT data set, 
and production of photographic images from appropriately- 
spaced sections. The next phase would be the contouring and 
digitization by anatomy experts, to define organs, tissues, and 
other structural entities in both CT and cryosectioned images. In 
terms of the two fundamental image representation methods 



discussed above, this phase would be the extraction of 
manipulable (geometric) objects from the pixel/sample based 
images obtained via radiography, magnetic resonance imaging, 
and photography. 

The image library thus constructed would be made available by 
the Library in both electronic formats (such as CD-ROM discs) 
and as photographic image sets. A wide range of educational, 
diagnostic, treatment planning, and commercial uses was 
predicted by the group. 

To address the issues raised by the workshop on 3-D 
Anatomical Reconstruction, the current panel representing both 
the producers and potential users of computer-based biomedical 
images was convened. Members of the panel undertook 
presentations and discussion to answer the following questions: 

1. What is the proper role for a public agency such as the 
NLM in relation to electronic imaging technologies? 
Should NLM change its current policy of exploring image 
library technologies to one of actually building and 
distributing image libraries? 

2. Would such image libraries, if available today or in the 
near future, be used sufficiently by students, health 
sciences educators, and/or practitioners to justify the 
expense involved in their creation and maintenance? 

3. Are new standards, data exchange conventions, or 
consortia arrangements needed to enhance the sharing of 
image data for purposes of education and health care? 

4. Is the proposed 3-D visible human project a good start 
for undertaking the development of an electronic image 
library? If so, how should this project be pursued? Are 
there other equally or more compelling image projects the 
NLM should consider? 
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Findings 

Digital Images: The panel notes that electronic imaging is having a major 

Clinical Applications impact on the conduct of clinical medicine. Particularly affected 

are the disciplines of neurology, reconstructive surgery 
(especially facial and oral surgery), neurosurgery, orthopedics, 
radiology, and radiation therapy. A growing emphasis on 3-D 
spatial information for treatment planning now supplements the 
traditional role of radiologic studies in diagnosis. 

In some specialties such as neurology, imaging studies serve as a 
screening procedure prior to requesting clinical consultation; 
clinical signs and symptoms are sometimes subordinated to 
imaging results in the management of patients, and referrals for 
asymptomatic abnormal imaging results are increasingly 
common. Stereotactic guidance for neurosurgical procedures 
using 3-D imaging data is under development. 

Management of traumatic, degenerative and congenital bony 
abnormalities, be they located in the head, thorax, or major 
joints, now often includes pre-operative 3-D imaging as an 
adjunct to therapy planning. The uses of imaging in this area 
include visual displays intended to convey a clear anatomic 
model to the physician. In addition, computer imaging data can 
be linked to robotically controlled milling machines to produce 
models of pathologic tissue for practice operations, to build 
prosthetic devices to guide implantation and reconstruction, and 
to guide the selection and/or remodelling of artificial joint 
hardware to fit a particular patient. Industrial research is 
currently in progress to use such data to guide the actual robotic 
milling of bone and joint surfaces in vivo. 

Additional therapy-planning uses of volumetric digital imaging 
include the matching of donor organ and recipient cavity sizes in 
cases of heart and liver transplant, the preoperative sizing of 
malignant tumors such as musculoskeletal sarcomas, and 
radiation therapy planning from 3-D volumetric tumor models. 

Digital Images: The panel recognizes that 3-D image reconstruction by 

Educational computer provides a new and promising method for the 

Applications education of health professionals. The understanding of complex 

structure-function relationships such as those in the central 
nervous system is a challenging educational problem which is 
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only pcx)rly solved by two dimensional images, physical models, 
and cadaver dissections. The educational goal is the 
development of a clear three dimensional model in the mind of 
the student, and the integration of structural knowledge with 
physiological and biochemical function. 

Just as the practicing surgeon might view abnormal, patient 
specific anatomy at a radiological workstation, the panel noted 
that a student's workstation would allow the display, rotation, 
selective "dissection" and reassembly of normal human anatomy 
by the student, controlled by graphical pointing devices and 
easy-to-use interface methods. Such methods would also greatly 
facilitate the understanding of congenital abnormalities and aid 
the planning of their corrections by rendering the existing 
collections of human embryos (e.g., Carnegie Collections, 
Blechschmidt Collection) in computer readable form. Initial 
work to develop both geometrical and image-based teaching files 
and control software is under way at several university centers. 

Digital Images: Biomedical research presents special requirements for 

Research Applications digital imaging. The need to correlate biochemical and 

physiologic data with anatomic location is especially acute in the 
neurosciences, where the recent discovery of large numbers of 
neurotransmitters and investigational techniques such as 
monoclonal antibody staining are generating a flood of new 
information which requires spatial mapping and correlation 
within the nervous systems of man and model systems such as 
rat brain. The tools for manipulating and sharing laboratory data 
add special requirements: the need to develop standardized 
formats for exchange of text, numeric, and graphical data among 
investigators; the need for user interface software which allows 
an investigator to easily view, edit, and compare the data from 
biochemical and neurocircuitry mapping experiments. Unlike the 
educational requirements for consensus standard image sets, 
research requirements emphasize the variability and one-of-a- 
kind nature of particular experiments. 

Research imaging requirements for a National Neural Circuitry 
Database are the object of a study currently being funded by the 
Alcohol, Drug Abuse and Mental Health Administration 
(ADAMHA) and conducted by the Institute of Medicine. The 
panel recognizes the importance of coordinating digital imaging 
projects among the federal agencies which sponsor biomedical 
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research, and making available the knowledge resulting from that 
research. 



Digital Images and The complex images of biology and medicine, as represented by 

Computer Networks digital volumetric data sets, constitute a challenge to currently 

available computer networks. Image files may range from 
hundreds of thousands to many millions of bits representing 
measures of volume (voxels) or points on a display screen 
(pixels). This contrasts with NLM's current bibliographic 
services where a database query may yield tens to a few 
thousands of alphanumeric characters, and wide area commercial 
networks providing 120 to 240 characters per second over voice- 
grade phone lines give acceptable response times. The 
transmission speed (bandwidth) necessary to accommodate 
image data will require that wide area networks be available 
with a capacity which equals or exceeds that currendy found in 
local area networks (10-100 megabits/second). 



Digital Imaging and The panel identifies a number of salient issues which bear 

the NLM upon National Library of Medicine involvement in digital image 

technologies and libraries. The most important of these are 

listed here. 



Acquiring and providing access to digital image libraries is 
entirely consistent with the NLM's institutional mandate to 
acquire, organize, and make available the knowledge of medicine 
and biology. No such publicly available digital libraries 
currendy exist. Clearly, the questions about this technology 
revolve around "when" and "how" the Library should proceed, 
not "if the Library should proceed. 

As noted by NLM's previous advisors, the greatest potential 
impact of NLM projects appears to be in the area of health 
professions education, where the establishment of image libraries 
representing normals and archetypal abnormals is an activity 
which cannot reasonably be undertaken by single academic 
institutions, and where there are few economic drivers in place 
to promote commercial development. However, educational 
institutions at the present time generally lack both the computer 
hardware of sufficient capacity, and the professional expertise 
among faculty to use 3-D computer images as a component of 
the curriculum. Indeed, the current emphasis upon molecular 
biological mechanisms of health and disease has led to a de- 



emphasis of gross anatomy in the curriculum, and fewer hours 
dedicated to "structural biology." Faculty training and 
acceptance of new modalities such as computer-generated images 
is critical to the success of this technology in the educational 
process. 

From a knowledge representation viewpoint, key issues remain 
in the development of methods to link spatial data-images and 
objects within images~to the symbolic data comprising the 
names, hierarchies, principles and theory which are the text- 
based understanding of visual things. The research group at the 
University of Washington has coined the term "Structural 
Informatics" to describe this field. Standards do not currently 
exist for such linkages among different data types, but this area 
of endeavor is new, rapidly changing, and the panel believes it is 
premature to insist upon such standards. In addition, much basic 
research is needed in the description and representation of 
morphological structures, and the connection of structural- 
anatomical to functional-physiological knowledge through the 
use of alternative modeling methods. Generalizable methods 
which support a hypertext-like model, where words can be used 
to find pictures, and pictures can be used as an index into 
relevant text, are needed. 

Libraries of normal structures should be created so that they can 
be easily Unked to related collections of diseased and abnormal 
structure, as well as those representing temporal variation (e.g., 
embryology). Research support by the NLM at key academic 
centers would assist in advancing the field, as would the 
convening of meetings to develop standards for data exchange 
and knowledge base sharing. 

It is clear that the current environment is one of rapid change 
and new development in computers and algorithms designed for 
graphics manipulation and rendering. For this reason, the panel 
believes that NLM emphasize the creation, standardization, 
exchange and distribution of volumetric image databases, rather 
than the development of applications software which uses those 
databases. 
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Recommendations 

The panel recommends the following: 

1. NLM should proceed with the proposed project to acquire 
and make available an image dataset of a complete, 
carefully selected male and female cadaver using the 
modalities of CT, MR, and photographs of cryosections. 
There must be a first project, and the panel believes this 
one to be quite appropriate. It will be a project through 
which standards will be developed and set for the 
electronic image libraries to come. It will provide a 
cornerstone and point of reference for future image 
collections which are related but extend along various 
conceptual axes: diseases of body parts, normal and 
abnormal growth and development, and dynamic temporal 
processes such as normal and abnormal body part 
motions. 

Guidelines for proceeding with this project are offered in 
Appendix A. 

2. The NLM should support a follow-on research effort to 
develop methods, tools and standards for classification of 
anatomic image data from the Visible Human Project, so 
that applications may be developed which can extract, 
manipulate and display image subsets on the basis of 
organs, tissues, body systems and biologic function. 
NLM should support the development of object 
classification techniques, standards of rendering (e.g., 
degree of photoreahsm, how lit, how colored, how 
antialiased, etc.) for this subsequent phase of the project 
in which geometric representations are developed from 
the initial image data sets. NLM should develop detailed 
resource requirements based on its proposed standards, to 
determine the timing and feasibility of proceeding with 
the comprehensive object classification of the initial 
image data. 



NLM should expand upon initial image libraries 
composed of normal structure to encompass specialized 
image collections which represent related structural 
information, such as embryological development, normal 
and abnormal variations, and disease-related images. In 
this regard, NLM should collaborate with appropriate 
professional societies and other organizations to identify 
and pursue worthy examples of specialized image 
collections which have been developed by subject 
experts. The dissemination and use of these digital 
library collections should be promoted. 

NLM should encourage and support investigator-initiated 
research into methods for representing and linking spatial 
and textual information (and other relevant datatypes), 
and support efforts to introduce computer reconstructed 
anatomical imaging technologies into health professions 
curricula nationwide. 

Fertile areas for grant support include: 

1) Structural Informatics 

2) Computer Graphics Technology as applied to 
biomedical images 

3) Basic biomedical research applications 

a. Developmental Biology, Embryology 

b. Neurobiology 

c. Cell and Tissue Biology 

d. Molecular and supramolecular structure 

4) CUnical applications 

a. Stereotactic procedures 

b. Radiation therapy 

c. Anesthesiology 

d. Radiology 

e. Organ systems imaging 

f. Orthopaedics 

5) Center Grants for computer-based imaging 

6) Program Projects 

7) Instrumentation grants 
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5. The NLM should develop and enhance its wide area 
computer network connections to provide an efficient 
electronic distribution mechanism for large digital files 
such as those encoding biomedical images. In the 
development phase of imaging projects, NLM should 
continue and enhance its connectivity to NSFnet and the 
research Internet. NLM should consider developing 
image-based applications which make use of the proposed 
gigabit speed National Research Network. 



Electronic Imaging, page 18 



Resource Recommendations 

The panel recognizes that its recommendations 
include a specific first project in building digital image 
libraries, and a more general endorsement of support for 
research to enable the fullest usage of such libraries. In 
the short term, the costs of undertaking the data 
acquisition phase of the Visible Human Project can be 
derived from existing technologies and similar projects 
undertaken on a smaller scale. The panel estimates that 
Phase I of this project would require approximately $1 
million and yield a uniquely valuable image data set. 

Phase II Phase n of the Visible Human Project-the classification 

of the image data-is considerably more labor and time 
intensive if undertaken using current manual or semi- 
automated methods, such as contour tracing by skilled 
anatomists. Between 40 and 50 man-years of effort 
would be required, at a cost of approximately $5 million, 
to fully classify the anatomic data. 

The panel encourages the NLM to provide research 
support to develop automated image processing tools for 
improving the speed and efficiency of biomedical image 
classification. Research to develop new methods for 
representing and linking structural and symbolic data in 
the life sciences is also needed. Support for 5-10 high 
quality investigator-initiated grants annually would 
greatly facilitate progress in these areas, at an estimated 
cost of $3 million. 

Computer Networks NLM connection to high speed computer networks is an 

extension of its current connections to NSFnet and the 
research Internet through the Lister Hill Center. The 
panel believes that over the next several years 
incremental improvement in bandwidth should be 
available to the NLM at approximately the same annual 
cost as it now bears (approximately $100,000 per year). 
Experimental projects involving commercial 
telecommunications networks should be pursued under 
collaborative cost-sharing agreements. 



Visible Human Project 
Phase I 



Structural Informatics 
Research 
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Appendix A: The Visible Human Project 

The panel recommends that a cooperative working group be formed 
from respondents to a competitive solicitation or request for proposals. 
The working group would be made up of representatives from several 
different institutions whose strengths and capabilities complement one 
another and make the Visible Human project feasible at a reasonable 
cost, with the greatest potential for widespread benefit to the biomedical 
community. 

The working group would be formed from successful applicants who 
develop proposals to work on one or more of the several tasks outlined 
below. It is understood that no single group would necessarily perform 
all of the tasks working in isolation as effectively and efficiently as a 
cooperative group who work on various tasks and coordinate their 
efforts. This could be done through either a contract or grant 
mechanism, but it is understood that not all of the costs would 
necessarily be borne by the NLM. If groups had existing support for 
similar or related work which could be adapted to these requirements, or 
their institutions would co-fund and underwrite some of the 
development costs, this would be most effective. 

Stage I: Acquisition of "Raw" Image Data 

The first step avoids all issues of classification by not identifying 
specific objects within the image data sets. It is therefore far short of 
clinical application, but guarantees feasibility and reliability. It provides 
the "raw truth" on which all subsequent elaborations can be built. 

Task 1. Obtain a set of sample adult human cadavers, three each 
of males and females. These cadavers must be "fresh" or 
very well preserved with minimal structural abnormality 
at the time of death. They should be young or middle- 
aged and fall within norms for size. 

Task 2. The preparation of the bodies must be done to fulfill 

stringent requirements for registration of serial sections 
obtained by medical imaging techniques or through serial 
sectioning. The bodies should be prepared with injection 
of radiodense fluid plastic to adequately define the 
arterial tree and fill the major vessels. Reference markers 
are required in an external plaster cast which should be 
applied to the specimen before processing. The required 
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preparation is a separate task, and may be performed by 
specially trained experts, not necessarily from the same 
site which provided the cadaver initially. 

Task 3. Whole body imaging of the prepared specimens is 
performed at 1 mm. intervals, using computed 
tomography of the whole body and magnetic resonance 
imaging of the head. This should be performed on all six 
cadavers soon after their demise. The cadavers should be 
rigidly fixed spatially prior to CT and MR imaging, such 
as by a total-body rigid cast, to prevent any motion or 
gravity-induced deformation of fresh tissues. Data from 
these imaging sequences would be archived to optical 
disks in a working group specified format that would be 
readable by NLM and other members of the group. All 
cadavers must be frozen after imaging is completed. It is 
anticipated that the pre-specified MRI sequences would 
require one or two hours of scanner time, while the CT 
scanning process to be performed under working group- 
specified conditions would require twelve hours or more, 
including archiving. 

The working group panel would review all images from 
the six cadavers at a collaborative meeting and select a 
single male specimen and single female specimen. The 
four remaining specimens might be used later in whole or 
in part for additional subprojects, or as a potential 
replacement in the event of an unforeseen problem with 
the ones selected first. 

Task 4. Anatomic sectioning would be performed at submillimeter 

intervals with color photographs taken at each slice level. 
These photographs would be obtained on pin-registered 
color film on as large a color transparency as possible 
(ideally 4 x 5 or 8 x 10 inch), and might be printed or 
copied as part of theu- distribution. The color images 
should be represented in an appropriate archival form, 
such as three color separation onto black and white fihn. 
The photographs should be made using a color film with 
a normal rather than enhanced color balance, such as 
Kodak EPN-100 Ektachrome Professional 100 ASA sheet 
film. This is the current standard for anatomic 
photography. 
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Task 5. The photographic images would be archived, with a 

recognized color standard (e.g., MacBeth, Kodak Q-60C 
Paper Reproduction Guide, Kodak Q-50C Kodachrome 
Reproduction Guide for gray scale) in each photo. These 
photographs would be digitized at 300 pixels/inch 
(professional photography/publishing resolution), or 2400 
X 3000 pixels, in RGB with 12 bits per color channel (R, 
G and B). They would also be archived at the same 
pixel density as the CT slices (either 512 x 512 or 1024 x 
1024) to correspond as precisely as possible with the 
pixels of the CT scans. This resolution can be derived 
from the high resolution scans without redigitizing. 

All digital data would be archived to optical disks in an 
appropriate format; the specification of this format should 
be obtained by consultation with experts in computer 
graphics and imaging. At this point, the entire working 
group would have access to all slice digital data and 
NLM would assist with distribution of it, after suitable 
reformatting to video disk, CD-ROMs, or other media as 
deemed practical and useful. A product would be 
available at this stage for outside groups, as the first 
resuh from the Visible Human Project. Outside groups 
could obtain the serial slice images and might use them 
in research or teaching. 

Stage II: Classification into Objects 

Task 1. Object definition would be performed on the slice data 

with segmentation, either manually or automatically. 
Quality control procedures and acceptance standards for 
the object definition results would be established by the 
working group and applied to the data sets at proof of 
performance. The object definition task may be restricted 
to a single body region or organ as proposed by each 
respondent to the solicitation. Ultimately, the entirety of 
the data sets should be segmented, but it is necessary that 
standards be established first, and in the early work 
associated with this initiative, it is understood that only 
specialized regions may be defined. 
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The object definitions would be distributed in the 
working group specified format, and could be made 
available expeditiously to outside groups by the NLM. 
The computer software tools used to implement the object 
definition would be available to accompany the raw slice 
data and object definitions. This should facilitate the 
generation of additional object definitions by outside 
groups who are studying specific areas of the body. 

Task 2. A hierarchical data base organization and associated 

retrieval functionality is required for the management of 
the very large amount of image data created in the 
Visible Human Project. The specification and 
implementation of the data base organization is a separate 
research task, and a successful outcome would involve a 
working retrieval system that operates at multiple levels 
and provides ties to additional sets of imagery such as 
nuclear medicine, ultrasound, histologic slice images, and 
text. 
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